Reinforced Labels: Multi-Agent Deep Reinforcement Learning for Point-Feature Label Placement

Over the recent years, Reinforcement Learning combined with Deep Learning techniques has successfully proven to solve complex problems in various domains, including robotics, self-driving cars, and finance. In this paper, we are introducing Reinforcement Learning (RL) to label placement, a complex task in data visualization that seeks optimal positioning for labels to avoid overlap and ensure legibility. Our novel point-feature label placement method utilizes Multi-Agent Deep Reinforcement Learning to learn the label placement strategy, the first machine-learning-driven labeling method, in contrast to the existing hand-crafted algorithms designed by human experts. To facilitate RL learning, we developed an environment where an agent acts as a proxy for a label, a short textual annotation that augments visualization. Our results show that the strategy trained by our method significantly outperforms the random strategy of an untrained agent and the compared methods designed by human experts in terms of completeness (i.e., the number of placed labels). The trade-off is increased computation time, making the proposed method slower than the compared methods. Nevertheless, our method is ideal for scenarios where the labeling can be computed in advance, and completeness is essential, such as cartographic maps, technical drawings, and medical atlases. Additionally, we conducted a user study to assess the perceived performance. The outcomes revealed that the participants considered the proposed method to be significantly better than the other examined methods. This indicates that the improved completeness is not just reflected in the quantitative metrics but also in the subjective evaluation by the participants

Comparison of Semantic Segmentation Approaches for Horizon/Sky Line Detection

Horizon or skyline detection plays a vital role towards mountainous visual geo-localization, however most of the recently proposed visual geo-localization approaches rely on \textbf{user-in-the-loop} skyline detection methods. Detecting such a segmenting boundary fully autonomously would definitely be a step forward for these localization approaches. This paper provides a quantitative comparison of four such methods for autonomous horizon/sky line detection on an extensive data set. Specifically, we provide the comparison between four recently proposed segmentation methods; one explicitly targeting the problem of horizon detection\cite{Ahmad15}, second focused on visual geo-localization but relying on accurate detection of skyline \cite{Saurer16} and other two proposed for general semantic segmentation -- Fully Convolutional Networks (FCN) \cite{Long15} and SegNet\cite{Badrinarayanan15}. Each of the first two methods is trained on a common training set \cite{Baatz12} comprised of about 200 images while models for the third and fourth method are fine tuned for sky segmentation problem through transfer learning using the same data set. Each of the method is tested on an extensive test set (about 3K images) covering various challenging geographical, weather, illumination and seasonal conditions. We report average accuracy and average absolute pixel error for each of the presented formulation.Comment: Proceedings of the International Joint Conference on Neural Networks (IJCNN) (oral presentation), IEEE Computational Intelligence Society, 201

Ink-and-Ray: Bas-Relief Meshes for Adding Global Illumination Effects to Hand-Drawn Characters

We present a new approach for generating global illumination renderings of hand-drawn characters using only a small set of simple annotations. Our system exploits the concept of bas-relief sculptures, making it possible to generate 3D proxies suitable for rendering without requiring side-views or extensive user input. We formulate an optimization process that automatically constructs approximate geometry sufficient to evoke the impression of a consistent 3D shape. The resulting renders provide the richer stylization capabilities of 3D global illumination while still retaining the 2D handdrawn look-and-feel. We demonstrate our approach on a varied set of handdrawn images and animations, showing that even in comparison to ground truth renderings of full 3D objects, our bas-relief approximation is able to produce convincing global illumination effects, including self-shadowing, glossy reflections, and diffuse color bleeding

Perception Motivated Hybrid Approach to Tone Mapping

We propose a general hybrid approach to the issue of reproduction of high dynamic range images on devices with limited dynamic range. Our approach is based on combination of arbitrary global and local tone mapping operators. Recent perceptual studies concerning the reproduction of HDR images have shown high importance of preservation of overall image attributes. Motivated by these findings, we apply the global method first to reproduce overall image attributes correctly. At the same time, an enhancement map is constructed to guide a local operator to the critical areas that deserve enhancement. Based on the choice of involved methods and on the manner of construction of an enhancement map, we show that our approach is general and can be easily tailored to miscellaneous goals of tone mapping. An implementation of proposed hybrid tone mapping produces good results, it is easy to implement, fast to compute and it is comfortably scalable, if desired. These qualities nominate our approach for utilization in time-critical HDR applications like interactive visualizations, modern computer games, HDR image viewers, HDR mobile devices applications, etc

Comaparing image-processing operators by means of visible differences predictor

Utilization of non-photorealistic techniques (NPR) is beneficial in many cases, in comparison with the use of tradi-tional rendering methods in computer graphics. The observer's sensation is often straighter, clearer, or even more valuable. There exists plenty of various NPR techniques in computer graphics, however application of one tech-nique to the specific problem is not necessarily as providential as usage of another one. There arises a strong need to classify NPR techniques with respect to their applicability, to find a mechanism able to compare NPR techniques automatically. The search for such a mechanism will be a long term goal. We present our first steps towards the solution to this problem — the comparison of 2D-based NPR techniques using Daly's Visible Differences Predictor (VDP). Results from our experiments are reported and will be used to improve communication in graphical user interfaces

Fft and convolution performance in image filtering on gpu

Many contemporary visualization tools comprise some image filtering approach. Since image filtering approaches are very computationally demanding, the acceleration using graphics-hardware (GPU) is very desirable to preserve interactivity of the main visualization tool itself. In this article we take a close look on GPU implementation of two basic approaches to image filtering – Fast Fourier Transform (frequency domain) and convolution (spatial domain). We evaluate these methods in terms of the performance in real time applications and suitability for GPU implementation. Convolution yields better performance than Fast Fourier Transform (FFT) in many cases; however, this observation cannot be generalized. In this article we identify conditions under which the FFT gives better performance than the corresponding convolution and we assess the different kernel sizes and issues of application of multiple filters on one image

Evaluation of two principal approaches to objective image quality assessment

Nowadays, it is evident that we must consider human perceptual properties to visualize information clearly and efficiently. We may utilize computational models of human visual systems to consider human perception well. Image quality assessment is a challenging task that is traditionally approached by such computational models. Recently, a new assessment methodology based on structural similarity has been proposed. In this paper we select two representative models of each group, the Visible Differences Predictor and the Structural SIMilarity index, for evaluation. We begin with the description of these two approaches and models. We then depict the subjective tests that we have conducted to obtain mean opinion scores. Inputs to these tests included uniformly compressed images and images compressed non-uniformly with regions of interest. Then, we discuss the performance of the two models, and the similarities and differences between the two models. We end with a summary of the important advantages of each approach. 1

Experimental system for visualisation of the light load

This paper presents our work on an experimental system for visualisation of the light load. The light load is defined as the total amount of light radiation received by all areas of a 3D scene. The emphasis of the presented system lays on outdoor architectural scenes, but indoor scenes are handled as well. The aim of the system is to visualise the light load either in a single moment or integrated during a longer time period. We have selected a hierarchical Monte Carlo radiosity method to solve the specified problem. This method was extended to handle parallel light sources and specular reflections. New “lighting” iteration was added to computation phase to consider natural light sources such as Sun and sky. The system was implemented in Java programming language using the Java3D API. Thanks to this implementation environment, our system is flexible, easy to modify and extend and it is suitable for experimental and educational purposes